Information Theoretic Approaches to Rincipal and Independent Component Analysis: a Simplified View
نویسنده
چکیده
Principal Component Analysis (PCA) and Independent Component Analysis (ICA) are the techniques that deal with extracting the independent components from linear mixtures of Gaussian and non-Gaussian data at the input respectively. PCA is a classical method that deals with the second order statistics of data. It is also known as Karhunen-Loeve Transform or the Hotelling Transform in some application areas. ICA is a generalization of PCA that takes into account the higher order statistics also. This paper presents a simplified view of information theoretic approaches to the problem of Principal Component and Independent Component Analysis. With the help of these techniques we find a linear representation of multivariate data so that the components are as statistically independent as possible. Such representations capture essential features of the data in many applications. This paper summarizes major approaches that are based on information theoretic concepts and includes the applications of ICA.
منابع مشابه
Combination of real options and game-theoretic approach in investment analysis
Investments in technology create a large amount of capital investments by major companies. Assessing such investment projects is identified as critical to the efficient assignment of resources. Viewing investment projects as real options, this paper expands a method for assessing technology investment decisions in the linkage existence of uncertainty and competition. It combines the game-theore...
متن کاملThe Effect of Grammar vs. Vocabulary Pre-teaching on EFL Learners’ Reading Comprehension: A Schema-Theoretic View of Reading
This study was designed to investigate the effect of grammar and vocabulary pre-teaching, as two types of pre-reading activities, on the Iranian EFL learners’ reading comprehension from a schema–theoretic perspective. The sample consisted of 90 female students studying at pre-university centers of Isfahan. The subjects were randomly divided into three equal-in-number groups. They participated ...
متن کاملRank based Least-squares Independent Component Analysis
In this paper, we propose a nonparametric rank-based alternative to the least-squares independent component analysis algorithm developed. The basic idea is to estimate the squared-loss mutual information, which used as the objective function of the algorithm, based on its copula density version. Therefore, no marginal densities have to be estimated. We provide empirical evaluation of th...
متن کاملMutual information minimization: application to Blind Source Separation
In this paper, the problem of Blind Source Separation (BSS) through mutual information minimization is addressed. For mutual information minimization, multi-variate score functions are first introduced, which can be served to construct a non-parametric “gradient” for mutual information. Then, two general gradient based approaches for minimizing mutual information in a parametric model are prese...
متن کاملA review on EEG based brain computer interface systems feature extraction methods
The brain – computer interface (BCI) provides a communicational channel between human and machine. Most of these systems are based on brain activities. Brain Computer-Interfacing is a methodology that provides a way for communication with the outside environment using the brain thoughts. The success of this methodology depends on the selection of methods to process the brain signals in each pha...
متن کامل